Learning Implicit Templates for Point-Based Clothed Human Modeling

We present FITE, a First-Implicit-Then-Explicit framework for modeling human avatars in clothing. Our framework first learns implicit surface templates representing the coarse clothing topology, and then employs the templates to guide the generation of point sets which further capture pose-dependent clothing deformations such as wrinkles. Our pipeline incorporates the merits of both implicit and explicit representations, namely, the ability to handle varying topology and the ability to efficiently capture fine details. We also propose diffused skinning to facilitate template training especially for loose clothing, and projection-based pose-encoding to extract pose information from mesh templates without predefined UV map or connectivity. Our code is publicly available at https://github.com/jsnln/fite.Comment: Accepted to ECCV 202

DiffuStereo: High Quality Human Reconstruction via Diffusion-based Stereo Using Sparse Cameras

We propose DiffuStereo, a novel system using only sparse cameras (8 in this work) for high-quality 3D human reconstruction. At its core is a novel diffusion-based stereo module, which introduces diffusion models, a type of powerful generative models, into the iterative stereo matching network. To this end, we design a new diffusion kernel and additional stereo constraints to facilitate stereo matching and depth estimation in the network. We further present a multi-level stereo network architecture to handle high-resolution (up to 4k) inputs without requiring unaffordable memory footprint. Given a set of sparse-view color images of a human, the proposed multi-level diffusion-based stereo network can produce highly accurate depth maps, which are then converted into a high-quality 3D human model through an efficient multi-view fusion strategy. Overall, our method enables automatic reconstruction of human models with quality on par to high-end dense-view camera rigs, and this is achieved using a much more light-weight hardware setup. Experiments show that our method outperforms state-of-the-art methods by a large margin both qualitatively and quantitatively.Comment: Accepted by ECCV202

Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Rendering

We present Tensor4D, an efficient yet effective approach to dynamic scene modeling. The key of our solution is an efficient 4D tensor decomposition method so that the dynamic scene can be directly represented as a 4D spatio-temporal tensor. To tackle the accompanying memory issue, we decompose the 4D tensor hierarchically by projecting it first into three time-aware volumes and then nine compact feature planes. In this way, spatial information over time can be simultaneously captured in a compact and memory-efficient manner. When applying Tensor4D for dynamic scene reconstruction and rendering, we further factorize the 4D fields to different scales in the sense that structural motions and dynamic detailed changes can be learned from coarse to fine. The effectiveness of our method is validated on both synthetic and real-world scenes. Extensive experiments show that our method is able to achieve high-quality dynamic reconstruction and rendering from sparse-view camera rigs or even a monocular camera. The code and dataset will be released at https://liuyebin.com/tensor4d/tensor4d.html

DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior

We present DreamCraft3D, a hierarchical 3D content generation method that produces high-fidelity and coherent 3D objects. We tackle the problem by leveraging a 2D reference image to guide the stages of geometry sculpting and texture boosting. A central focus of this work is to address the consistency issue that existing works encounter. To sculpt geometries that render coherently, we perform score distillation sampling via a view-dependent diffusion model. This 3D prior, alongside several training strategies, prioritizes the geometry consistency but compromises the texture fidelity. We further propose Bootstrapped Score Distillation to specifically boost the texture. We train a personalized diffusion model, Dreambooth, on the augmented renderings of the scene, imbuing it with 3D knowledge of the scene being optimized. The score distillation from this 3D-aware diffusion prior provides view-consistent guidance for the scene. Notably, through an alternating optimization of the diffusion prior and 3D scene representation, we achieve mutually reinforcing improvements: the optimized 3D scene aids in training the scene-specific diffusion model, which offers increasingly view-consistent guidance for 3D optimization. The optimization is thus bootstrapped and leads to substantial texture boosting. With tailored 3D priors throughout the hierarchical generation, DreamCraft3D generates coherent 3D objects with photorealistic renderings, advancing the state-of-the-art in 3D content generation. Code available at https://github.com/deepseek-ai/DreamCraft3D.Comment: Project Page: https://mrtornado24.github.io/DreamCraft3D

Control4D: Dynamic Portrait Editing by Learning 4D GAN from 2D Diffusion-based Editor

Recent years have witnessed considerable achievements in editing images with text instructions. When applying these editors to dynamic scene editing, the new-style scene tends to be temporally inconsistent due to the frame-by-frame nature of these 2D editors. To tackle this issue, we propose Control4D, a novel approach for high-fidelity and temporally consistent 4D portrait editing. Control4D is built upon an efficient 4D representation with a 2D diffusion-based editor. Instead of using direct supervisions from the editor, our method learns a 4D GAN from it and avoids the inconsistent supervision signals. Specifically, we employ a discriminator to learn the generation distribution based on the edited images and then update the generator with the discrimination signals. For more stable training, multi-level information is extracted from the edited images and used to facilitate the learning of the generator. Experimental results show that Control4D surpasses previous approaches and achieves more photo-realistic and consistent 4D editing performances. The link to our project website is https://control4darxiv.github.io.Comment: The link to our project website is https://control4darxiv.github.i

Mutations in TUBB8 and Human Oocyte Meiotic Arrest

BACKGROUND Human reproduction depends on the fusion of a mature oocyte with a sperm cell to form a fertilized egg. The genetic events that lead to the arrest of human oocyte maturation are unknown. METHODS We sequenced the exomes of five members of a four-generation family, three of whom had infertility due to oocyte meiosis I arrest. We performed Sanger sequencing of a candidate gene, TUBB8, in DNA samples from these members, additional family members, and members of 23 other affected families. The expression of TUBB8 and all other β-tubulin isotypes was assessed in human oocytes, early embryos, sperm cells, and several somatic tissues by means of a quantitative reverse- transcriptase–polymerase-chain-reaction assay. We evaluated the effect of the TUBB8 mutations on the assembly of the heterodimer consisting of one α-tubulin polypeptide and one β-tubulin polypeptide (α/β-tubulin heterodimer) in vitro, on microtubule architecture in HeLa cells, on microtubule dynamics in yeast cells, and on spindle assembly in mouse and human oocytes. RESULTS We identified seven mutations in the primate-specific gene TUBB8 that were responsible for oocyte meiosis I arrest in 7 of the 24 families. TUBB8 expression is unique to oocytes and the early embryo, in which this gene accounts for almost all the expressed β-tubulin. The mutations affect chaperone-dependent folding and assembly of the α/β-tubulin heterodimer, disrupt microtubule behavior on expression in cultured cells, alter microtubule dynamics in vivo, and cause catastrophic spindle-assembly defects and maturation arrest on expression in mouse and human oocytes. CONCLUSIONS TUBB8 mutations have dominant-negative effects that disrupt microtubule behavior and oocyte meiotic spindle assembly and maturation, causing female infertility. (Funded by the National Basic Research Program of China and others.

Impact Analysis of Vanadium SPND Prompt Gamma Current in PWR-core Measurement

Vanadium self-powered neutron detector (SPND) is applied in Gen-Ⅲ commercial PWR AP1000 to simultaneously and continuously measure the in-core neutron flux density, and to control the safe and stable operation of nuclear reactors. However, not only neutron flux can interact with vanadium SPND, then generate the response current. But also, the gamma flux from fission, capture and decay reaction can generate the response current. And the prompt gamma current can impact the measured response current. The response current can reflect the operation condition based on an accurate SPND response current calculation. Thus, a SPND simulation analysis system was designed and developed based on the PWR-core analysis code system NECP-Bamboo code and Monte-Carlo code NECP-MCX. There are three principal steps in the simulation procedure of SPND response current, viz lattice simulation to obtain the nuclide composition and cross section of emitter and the few-group constants, SPND simulation to obtain electron escape probability and gamma sensitivity in different assemblies and burnup based on the conservation of electron number, core simulation to obtain the response current during core operation. The decay activity was employed to facilitate the accuracy based on micro-depletion. Based on this system, the response current and the gamma response current ratio of a vanadium SPND in the AP1000 core was simulated. The results were verified against measured values and perform well. The results show that the prompt gamma current ratio is different in different assemblies. Most of them can’t be ignored. It is necessary to consider the gamma response current in the response current simulation, since the absolute value of prompt gamma current ratio is above 6.3%. The impact of prompt gamma response current to some measurements in AP1000 core was analyzed. The error is large than 10% without gamma response current. Then, the impact of prompt gamma current on core axial flux deviation (AFD) measurement and online power reconstruction was analyzed. If the neutron current is only considered, the difference between the measured and calculated values of AFD is 1+α times (α is the ratio of prompt gamma response current to neutron response current), and the difference between the measured and calculated values of power reconstruction is 1/(1+α). The error of power reconstruction corrected by α performs well than the uncorrected one. Thus, it is necessary to use different α value in different assemblies to correct the response current

HDhuman: High-quality Human Performance Capture with Sparse Views

In this paper, we introduce HDhuman, a method that addresses the challenge of novel view rendering of human performers that wear clothes with complex texture patterns using a sparse set of camera views. Although some recent works have achieved remarkable rendering quality on humans with relatively uniform textures using sparse views, the rendering quality remains limited when dealing with complex texture patterns as they are unable to recover the high-frequency geometry details that observed in the input views. To this end, the proposed HDhuman uses a human reconstruction network with a pixel-aligned spatial transformer and a rendering network that uses geometry-guided pixel-wise feature integration to achieve high-quality human reconstruction and rendering. The designed pixel-aligned spatial transformer calculates the correlations between the input views, producing human reconstruction results with high-frequency details. Based on the surface reconstruction results, the geometry-guided pixel-wise visibility reasoning provides guidance for multi-view feature integration, enabling the rendering network to render high-quality images at 2k resolution on novel views. Unlike previous neural rendering works that always need to train or fine-tune an independent network for a different scene, our method is a general framework that is able to generalize to novel subjects. Experiments show that our approach outperforms all the prior generic or specific methods on both synthetic data and real-world data